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Abstract 


As  the  United  States  Department  of  Defense  continues  to  increase  the  number  of  Remotely 
Piloted  Aircraft  (RPA)  operations  overseas,  improved  Human  Systems  Integration  becomes  increasingly 
important.  Manpower  limitations  have  motivated  the  investigation  of  Multiple  Aircraft  Control  (MAC) 
configurations  where  a  single  pilot  controls  multiple  RPAs  simultaneously.  Previous  research  has 
indicated  that  frequent,  unpredictable,  and  oftentimes  overwhelming,  volumes  of  communication  events 
can  produce  unmanageable  levels  of  system  induced  workload  for  MAC  pilots.  Existing  human  computer 
interface  design  includes  both  visual  information  with  typed  responses,  which  conflict  with  numerous 
other  visual  tasks  the  pilot  performs,  and  auditory  information  that  is  provided  through  multiple  audio 
devices  with  speech  response.  This  paper  extends  previous  discrete  event  workload  models  of  pilot 
activities  flying  multiple  aircraft.  Specifically,  we  examine  statically  reallocating  communication 
modality  with  the  goal  to  reduce  and  minimize  the  overall  pilot  cognitive  workload.  The  analysis 
investigates  the  impact  of  various  communication  reallocations  on  predicted  pilot  workload,  measured  by 
the  percent  of  time  workload  is  over  a  saturation  threshold. 
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1.  Introduction 

Over  the  past  several  decades,  the  US  Air  Force  has  harnessed  and  exploited  the  immense  tactical 
power  that  middle  and  high-altitude  Remotely  Piloted  Aircraft  (RPAs)  bring  to  the  battlefield.  As  a 
consequence,  the  demand  for  RPA  operational  support  continues  to  increase.  It  is  important  to  realize  that 
RPAs  are  part  of  a  complex  system.  The  system  has  many  components  including  one  or  more  air 
vehicles,  ground  control  stations  (GCS)  for  both  primary  mission  control  and  takeoff/landing,  a  suite  of 
communications  (including  intercom,  chat,  radios,  phones,  a  satellite  link,  etc),  support  equipment,  and 
operations  and  maintenance  crews  [1].  Assets  and  requisite  resources  to  support  those  operations  are 
limited  and  personnel  resources,  particularly  RPA  pilots,  often  prove  a  nontrivial  constraint.  This 
inevitably  leads  innovators  to  seek  out  RPA  force-multiplying  efficiencies  to  assist  in  bridging  the 
resource/demand  gap.  One  such  efficiency  being  pursued  is  simultaneous  control  of  multiple  aircraft  by  a 
single  pilot,  or  Multi  Aircraft  Control  (MAC).  This  concept  of  operations  has  been  documented  in  the  US 
Air  Force  UAV  flight  Plan  [2]. which  calls  for  future  systems  in  which  a  single  pilot  will  simultaneously 
control  multiple  RPAs  to  enable  increased  aerial  surveillance  without  increasing  pilot  manpower 
requirements.  Previous  research  on  the  cognitive  workload  experienced  by  pilots  during  MAC  indicated 
that  frequent,  unpredictable,  and  oftentimes  overwhelming  volumes  of  communication  events  can  produce 
unmanageable  levels  of  system  induced  workload  for  MAC  pilots  [3].  To  further  investigate  this 
identified  problem,  our  study  makes  use  of  IMPRINT  Pro,  a  Multiple  Resource  Theory  (MRT)  based 
dynamic,  stochastic  simulation  to  analyze  impacts  to  cognitive  workload  by  a  disciplined  communication 
modality  reallocation  construct. 

2.  Background 

In  the  RPA  domain,  communication  is  a  continuous  and  demanding  process.  Crews  must  track,  at  a 
minimum,  information  regarding  weather,  threats,  mission  tasking,  mission  coordination,  target 
coordination,  airspace  coordination,  fleet  management,  and  status  and  location  of  any  friendly  units.  The 
RPA  pilot  is  not  only  responsible  for  aircraft  control  but  is  also  a  critical  member  in  a  multi-path 
communications  infrastructure  [4],  In  the  ground  station,  communication  with  the  pilot  takes  place  in 
one  of  two  modalities:  textual  chat  window) s)  or  the  speech-based  radio  systems.  At  any  given  moment,  a 
pilot  may  need  to  monitor  multiple  chat  windows  and  listen  to  numerous  parties  operate  over  the  radio. 
The  multitude  of  communication  sources  and  different  media  coupled  with  the  quick  inter-arrival  rate  of 
these  events  during  a  dynamic  scenario  drives  an  incredible  cognitive  workload  for  the  pilot. 

Cognitive  or  mental  workload  expresses  the  task  demands  placed  on  an  operator  [5],  Calculation  of 
task  demand,  or  task  load,  often  considers  the  goals  of  the  operator,  the  time  available  to  perform  the 
tasks  necessary  to  accomplish  the  goals,  and  the  performance  level  of  the  operator  [6],  Therefore, 
workload  increases  when  the  number  or  difficulty  of  tasks  necessary  to  perform  a  goal  increase,  or  when 
the  times  allotted  to  complete  these  tasks  decrease.  Assuming  that  the  operator  has  a  limited  amount  of 
mental  resources  (e.g.,  attention,  memory,  etc.)  that  he  or  she  can  utilize  to  complete  the  necessary  tasks, 
mental  workload  corresponds  to  the  proportion  of  the  operator’s  mental  resources  demanded  by  a  task  or 
set  of  tasks.  Several  methods  have  been  employed  to  measure  and  quantify  mental  workload  over  the  past 
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four  decades  and  have  been  summarized  in  numerous  publications  [5,7,8].  For  example,  [7]  summarizes 
secondary  task  measures  of  workload  where  a  subject  is  asked  to  perform  a  secondary  task  which  intrudes 
upon  the  primary  task.  The  secondary  task  performance  is  measured,  giving  an  indication  of  how  much 
mental  resources  the  subject  had  available  to  them.  If  the  secondary  task  performance  is  high,  it  is  taken 
as  an  indication  that  the  primary  task  induces  relatively  low  workload  on  the  subject.  This  measure  is  not 
ideal,  however,  because  it  is  subject  to  a  variety  of  mitigation  strategies,  which  could  skew  performance 
results.  The  measure  also  does  not  work  for  situations  with  many  tasks  where  overall  workload  measures 
are  desired.  Stand  alone  performance  measures  are  also  discussed  [7].  Stand  alone  measures  tend  to  be 
effective  for  single  tasks  but  cannot  be  extended  to  multiple  tasks.  Measuring  methods  also  tend  to  be 
intrusive  and  may  interfere  with  results.  The  current  analysis  incorporates  Multiple  Resource  Theory 
(MRT)  into  the  workload  calculations  to  account  for  channel  conflict  driven  workload.  MRT 

As  a  theory,  MRT  purports  the  existence  of  four  mental  dimensions  (or  channels)  available  to  process 
information  and  perform  tasks.  The  dimensions  include  processing  stages,  processing  codes,  perceptual 
modalities  and  visual  channels.  These  channels  are  allocated  to  concurrent  tasks  with  the  difficulty  of  the 
tasks  and  the  demand  conflict  between  channels  driving  the  overall  mental  workload  value  [9].  MRT 
accurately  describes  the  concurrent  nature  of  tasks  imposed  on  an  RPA  pilot  (performing  primary  tasks 
while  communicating  and  monitoring  communication)  and  is  therefore  an  appropriate  theory  to  apply  to 
the  present  analysis. 

3.  Method 

Therefore,  the  specific  channels  employed  by  the  modeled  communication  events  are  highly  relevant 
to  the  MRT  workload  calculations.  As  communication  events  begin  to  conflict  with  existing  work 
activities  on  the  various  channels,  the  calculated  overall  cognitive  workload  will  account  for  such 
conflicts.  This  construct  enables  the  analysis  to  address  the  question  of  whether  or  not  adjusting  the 
intentional  allocation  of  communication  events  to  particular  modalities  will  be  able  to  meaningfully  affect 
overall  cognitive  workload. 

3.1.  Model 

A  previous  model  of  pilot  mental  workload  [3]  was  utilized  to  understand  the  impact  of 
communications  modality.  This  model  employed  functional  analysis  and  task  allocation  to  construct  an 
executable  architecture  of  the  multiple  RPA  system.  This  architecture  was  then  replicated  within  the 
Improved  Performance  Research  Integration  Tool  (IMPRINT)  to  estimate  the  pilot’s  workload  under 
various  mission  segments,  such  as  handover,  transit,  emergency,  benign  and  dynamic  surveillance,  etc. 
IMPRINT  is  a  human  performance  tool  used  by  the  Army  Research  Lab  to  simulate  cognitive  tasks  and 
mental  workload,  as  well  as  manpower  and  maintenance  missions.  This  model  relied  on  subject  matter 
expert  input  to  develop  distributions  for  the  length,  frequency,  and  difficulty  of  the  events  that  induce 
workload  on  the  pilot.  The  original  research  on  this  model  indicated  that  workload  was  particularly  high 
during  what  were  termed  dynamic  mission  segments.  These  mission  segments  often  involve  high  levels 
of  communication  between  the  pilot  and  external  actors  to  facilitate  the  tracking  or  observation  of  moving 
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targets.  High  levels  of  communication  resulted  in  particularly  “high”  pilot  workload  while  operating  a 
single  aircraft  and,  “excessive”  workload  while  controlling  multiple  dynamic-mission  aircraft.  The 
original  research  indicated  that  a  reduction  in  pilot  workload  imposed  by  communication  would  be 
necessary  to  facilitate  MAC. 

To  understand  the  potential  impact  of  communication  modality  on  operator  workload,  the 
communications  portion  of  the  earlier  workload  model  was  modified  to  permit  communications  events  to 
be  reallocated  to  alternate  communications  modalities.  The  revised  model  permits  communication  events 
that  were  originally  allocated  to  the  auditory  channels  where  the  operator  listens  and  speaks  to  the  visual 
and  fine  motor  channels  where  the  operator  reads  and  types,  or  vice  versa. 

Figure  1  depicts  the  high  level  structure  of  the  revised  communications  model.  The  gray  boxes 
indicate  model  elements  that  were  added  to  facilitate  this  particular  evaluation.  Communication  events 
are  generated  with  a  mission  segment  dependent  frequency  and  their  interarrival  times  are  exponentially 
distributed.  In  the  original  model,  as  a  communication  event  is  generated,  it  is  assigned  as  either  an 
auditory  event  or  a  text-based  event  with  25%  of  the  events  being  allocated  as  auditory  events  and  the 
remaining  allocated  as  text  events.  Half  of  the  auditory  events  then  required  the  pilot  to  talk  or  listen 
while  90%  of  the  text  events  required  the  pilot  to  read  while  only  10%  of  the  events  required  the  pilot  to 
type  a  response. 


Fig.  1.  Modified  communication  model  of  pilot  workload 

To  conduct  the  current  evaluation,  the  model  was  modified  as  shown  above.  The  auditory  and  text 
events  shown  in  gray  have  the  potential  (through  a  notional  device  or  software)  to  either  pass  an  auditory 
or  text  event  as  a  respective  auditory  or  text  event  or  to  convert  an  auditory  event  to  a  text  event  or 
convert  a  text  event  to  an  auditory  event.  With  this  modification,  it  is  assumed  that  the  characteristics  of 
the  communication  are  due  to  communication  needs,  such  that  if  a  text  event  in  the  original  model  had  a 
90%  chance  of  providing  an  input  to  the  pilot  and  only  a  10%  chance  of  an  output  to  the  pilot,  a  text  event 
converted  to  an  auditory  event  has  a  90%  probability  to  require  the  pilot  to  listen  and  only  a  10% 
probability  to  require  the  pilot  to  talk.  The  parameters  V  (for  Voice  reallocation)  and  T  (for  Text 
reallocation)  provide  the  ability  to  convert  auditory  or  text  events  to  its  compliment.  If  V  and  T  are  both 
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100%,  the  revised  model  is  the  same  as  the  original  model.  Reducing  either  of  these  parameters  permits  a 
portion  of  one  type  of  communication  event  to  be  reallocated  to  the  complimentary  communication  event. 
Although  not  shown,  it  is  then  assumed  that  some  percentage  of  the  final  events  generate  a  repeat 
communication  event,  indicative  of  a  continued  conversation.  This  aspect  of  the  model  was  not  changed. 

3.2.  Experimental  Design 

For  this  paper,  a  total  of  six  “levels”  of  voice/text  allocation  were  selected  such  that  the  percent  of 
voice  communication  were  varied  between  0  and  100  percent.  For  levels  of  voice  communications  less 
than  25%,  V  was  varied  while  T  was  maintained  at  100%.  Flowever,  for  levels  of  voice  communications 
greater  than  25%,  V  was  maintained  at  100%  while  T  was  varied  to  achieve  the  desired  communications 
levels.  All  analysis  was  performed  for  a  10  hour  dynamic  mission  segment  with  a  single  pilot  operating 
the  aircraft.  Although  IMPRINT  does  not  currently  have  built-in  Monte  Carlo  functionality  for  the 
metrics  of  our  concern,  an  external  batch  application  was  developed  to  automate  replications.  A  total  of 
10  replications  for  each  of  six  levels  using  10  different  random  number  seeds  were  performed  to  gather 
the  output  data. 

The  output  of  the  IMPRINT  model  was  analyzed  to  determine  the  proportion  of  time  that  the 
operator  would  experience  workload  values  over  a  specified  task  saturation  threshold.  A  workload  value 
of  60  was  calibrated  to  be  about  the  90%  of  operator  “red-line”,  which  indicates  the  workload  value  a 
pilot  can  experience  without  degraded  performance  [10].  The  mean  and  variance  across  the  10 
replications  for  each  communication  ratio  was  calculated.  Analysis  of  Variance  (ANOVA)  and  the  Tukey 
post-hoc  tests  were  employed  determine  the  statistical  differences  between  the  average  of  percent  time 
over  threshold. 

4.  Results 

Figure  2  shows  the  percent  time  over  threshold  as  a  function  of  the  percentage  of  voice 
communication.  A  one  way  ANOVA  indicated  a  significant  effect  of  the  percent  of  voice  communication 
upon  the  percentage  of  time  over  threshold  (p  <  0.001).  As  shown  in  Figure  1,  the  percent  of  time  over 
threshold  is  reduced  as  the  percent  of  voice  communication  is  increased  from  0%  to  40%.  At  40%  voice 
communication  the  percent  time  over  threshold  is  reduced  to  24.5%  compared  to  33.1%  with  0%  voice 
communication.  This  change  is  statistically  significant.  The  change  in  percent  time  over  threshold  is 
statistically  insignificant  as  the  percent  of  voice  communication  is  increased  from  40%  to  60%.  This 
trend  indicates  that  pilot  workload  is  reduced  by  the  use  of  both  auditory  and  text-based  communications 
in  this  system. 
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Fig.  2.  Percent  Time  Over  Threshold  as  the  percentage  of  reallocated  voice  events 

Results  further  show  that  the  percent  time  over  threshold  is  greater  at  0%  voice  than  at  100%  voice 
communications.  This  might  have  been  expected  as  reading  and  typing  likely  conflicted  directly  with 
other  tasks  being  performed  by  the  pilot,  including  visually  monitoring  the  status  and  manipulating  the 
controls  of  the  RPAs.  As  such  workload  is  highest  when  all  of  the  communication  is  allocated  entirely  to 
the  visual  channel. 

5.  Conclusions 

The  model  indicates  that  by  deliberately  allocating  communication  between  auditory  and  text-based 
modalities  the  pilot’s  workload  and  particularly  the  percentage  of  time  the  pilot  operates  beyond  their  task 
saturation  red-line  can  be  statistically  reduced.  The  model  shows  that  the  percent  of  time  over  red-line  is 
greatest  when  all  of  the  communication  is  allocated  to  the  text-based  communications  such  that  zero 
percent  of  the  communication  is  allocated  to  voice.  This  type  of  communication  is  most  likely  to  conflict 
with  other  tasks  involving  the  visual  system  to  monitor  the  RPA  and  the  small  motor  system,  which  is 
used  by  the  pilot  to  control  the  RPA.  As  communication  events  are  moved  from  text  to  auditory,  the 
workload  decreases.  However,  as  more  communication  is  moved  to  the  auditory  channel,  the  percent  of 
mission  time  over  the  red-line  to  increases.  The  increase  likely  occurs  as  the  auditory  tasks  begin  to 
overlap  and  conflict  with  one  another  to  increase  workload.  There  appears  to  be  an  optimal  allocation  of 
communications  between  voice  and  text  modalities  to  achieve  the  lowest  workload  given  a  constant 
traffic  load.  Future  research  will  examine  dynamic  reallocation  of  modalities. 
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